Estimating Quantities: Comparing Simple Heuristics and Machine Learning Algorithms

Estimating quantities is an important everyday task. We analyzed the performance of various estimation strategies in ninety-nine real-world environments drawn from various domains. In an extensive simulation study, we compared two classes of strategies: one included machine learning algorithms such as general regression neural networks and classification and regression trees, the other two psychologically plausible and computationally much simpler heuristics (QEst and Zig-QEst). We report the strategies’ ability to generalize from training sets to new data and explore the ecological rationality of their use; that is, how well they perform as a function of the statistical structure of the environment. While the machine learning algorithms outperform the heuristics when fitting data, Zig-QEst is competitive when making predictions out-of-sample.